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Image Sensor 

The present invention relates to video image sensors, and more particularly to 
colour and light sensing in relation to such sensors. 

Red, green and blue (RGB) are three primary additive colours whereby 
individual components are added together to form a desired colour and this 
format is the basic colour space model which is widely used throughout 
imaging. In particular, broadcast, video and imaging standards make use of 
RGB signals to derive luminance and colour difference video signals, such as 
YUV, YIQ or YCbCr colour spaces. Colour spaces are mathematical 
representations of a set of colours. In YUV, Y represents the black and white 
information in the video signal (i.e. luminance), and U and V represent the 
colour information in the video signal (i.e. chrominance). The basic equations 
for converting between RGB and YUV are: 

V = 0.2997? + 0.587G + 0. 1 145 
U = -0A 47 R - 0.289G + 0.4365 
U = 0.492(5 - Y) 

V = 0.615/J-0.515G- 0.100B 

V = 0.877(7? - Y) 

In YIQ colour space, I stands for in-phase and Q stands for quadrature, which 
is the modulation method used to transmit the colour information; YIQ can be 
derived from YUV. YCbCr is a scaled and offset version of the YUV colour 
space. 

Currently, in known colour video sensing devices such as video cameras, 
black and white pixel sensors are adapted to read colour information by 
disposing colour filters over the sensors, which sensors typically are CCD or 
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CMOS sensors. A standard arrangement is for the pixels to be grouped in 2 x 
2 blocks, with diagonally opposite pixels being responsive to green light and 
the other two diagonally opposite pixels being responsive to blue and red light 
respectively. These are known as RGB filters. The reason why there are two 
5 green pixels is that more image information is present in green light. 

It has been noted that a problem of using such an arrangement of RGB filters 
is that it introduces a light attenuation of approximately 50:1 . The share of the 
pixels between green, blue and red filters means that there is only 25% of the 

10 black and white pixel sensors for each of the blue and red filters, and 50% for 
the green filters. The result is that the sensor loses a great deal of colour 
resolution. Attempts are made in the downstream processing of the sensor 
output to recover the original resolution and a common technique is 
interpolation of the sensor output information using complex proprietary 

15 algorithms. This in essence involves estimating by interpolation what the 
colour response might be at a given pixel location based on the sensor 
outputs from the pixels surrounding that given pixel. However, since 
interpolation is a very approximate calculation its effectiveness varies widely 
depending on the complexity of the algorithm. 

20 

In addition, use of RGB filters leads to a reduction in sensitivity. Furthermore, 
there is also a reduction in the spatial resolution of the sensor, RGB colour 
filtering reducing resolution approximately by a factor of four. 

25 A solution to the problem of loss of sensitivity due to RGB filters is to increase 
the length of time that the sensors are exposed to the scene to be captured 
on video. However, the knock-on effect of this is that the camera is more 
susceptible to shaking, and also blurring due to subject movement is more 
likely to occur. This is a particularly marked problem for CMOS sensors 

30 which are of much lower sensitivity than CCD sensors. Also, since sensor 




noise is additive, longer exposure periods results in higher noise floor, and 
thus the image signal is swamped. 

As mentioned previously, in general, resolution of the image is determined by 
5 interpolation. Since there are a greater number of pixels in the green colour 
band and since also green light contains greater image information, this is 
used to increase the effective resolution of red and blue sensors. The main 
problem with this technique is that it is highly dependent on the quality of the 
algorithm used and its complexity. Sometimes the quality of the red and blue 
10 information can be improved further using green information. 

The present invention is based on the observation that the human eye is not 
as sensitive to colours as it is to brightness. In light of this observation, it is 
proposed that instead of at the pixel level having red, green and blue filters 

15 mapping onto respective pixels, certain of the colour pixels be replaced by so- 
called 'grey' filters in which there are no colour filters associated with such 
pixels. Accordingly, the present invention resides in an image sensing device 
having an array of image sensors and a filter structure disposed in relation to 
the image sensors, the filter structure comprising blocks of filter groups, each 

20 group having a combination of one or more areas adapted so as to allow light 
to pass therethrough onto corresponding image sensors with colour filtering, 
and one or more areas adapted so as to allow light to pass therethrough onto 
corresponding image sensors without colour filtering. 

25 A key advantage of the present invention, is that because there are certain 
areas in the filter which will allow the incident light to pass through without any 
colour filtering, the attenuation of the light is greatly reduced and 
consequently sensitivity of the device is enhanced compared with known RGB 
filter image sensing devices. 

30 




A further advantage of the present is that it is better suited to image 
compression because it provides for better compatibility with such 
compression techniques. This is because image compression systems such 
as MPEG and JPEG have similar luminance and chrominance requirements 
5 to that output from the sensors of the present invention. In particular, such 
compression standards reduce chrominance, while sustaining the same level 
of luminance. Because the grey resolution and colour resolution of an image 
sensor of the present invention is designed to match those of MPEG and 
JPEG compression standards, the signals output from the sensors require a 
10 minimal amount of additional signal conditioning for JPEG and MPEG 
compression. 

A further advantage is that since there is a direct measure of luminance by 
virtue of those areas of the filter where there are no colour filters, the grey 
15 pixels of the present invention offer a full resolution of the image and therefore 
there is a significant improvement in image quality. 

Additionally the image sensor calculation of YUV is much simplified over 
known prior art systems. 

20 

The present invention will now be described by way only of example with 
reference to the accompanying drawings in which: 

Figure 1 schematically illustrates a first embodiment of the present invention; 
25 Figure 2 schematically illustrates a second embodiment of the present 
invention; and 

Figure 3 schematically illustrates a third embodiment of the present invention. 



30 



Referring initially to Figure 1, there is shown a filter group of a unitary block 
(10) from a repeating pattern of such blocks forming a filter structure in 
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accordance with a first embodiment of the present invention. The filter 
structure is overlaid in relation to an array of pixel sensors of a video imaging 
device. The block (10) consists of six filter areas. Four of these are areas in 
which there are no colour filters overlaying the respective pixels and these are 
5 denoted Y (for ease of reference these will be referred to as grey sensors). 
There is one area denoted B which represents a blue filter and there is 
another area denoted R which represents a red filter. The Y areas 
conveniently are formed by apertured windows in the filter structure which 
align with respective pixels sensors. In this embodiment the Y areas are 
10 generally square, although rectangular and other shapes are equally 
permissible, and they are arranged spaced from one another in a generally 
square configuration. The blue and red filters are disposed centrally in 
relation to the four square grey sensors and conveniently are triangular in 
shape, although again other shapes are equally permissible. 

15 

The process involved in capturing a video image using the filter structure of 
the present invention begins by first starting exposure for the red and blue 
filters. As mentioned above, since chrominance filters are less sensitive to 
light than luminance filters it is necessary to expose them for longer exposure 

20 times than luminance filters. Sensor readings are then taken from the four 
grey sensors by sampling them. While reading the luminance response the 
chrominance filters are subjected to extended exposure time. Next, the red 
and blue pixels are sampled. This series of steps is repeated for each block 
and involves reading a row of pixel sensors, resetting and looping. Typically, 

25 the refresh rate is 30 frames per second and is sequential. 

Since in known RGB arrangements the green filters occupy 50% of the filter 
space in a 2 x 2 block, it is proposed that in this embodiment the Y areas also 
occupy half of the filter space in a block, and because there are a group of 
30 four Y areas in each block, this can be expressed as fraction 4/8 of the overall 
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filter space in each block. As there is no colour filtering in these Y areas the 
grey sensors are 50 times more sensitive and thus the overall sensitivity of 
the block is increased by 4/8 x 50 giving 25 times the sensitivity of prior art 
RGB filters. It should be mentioned that in order to achieve the same overall 
5 area of filter block as RGB filters, as there are four Y areas in place of two 
green filters, the size oteachxof the Y areas is reduced by 50% in comparison 
to green filters. J 

It was explained above that the blue and red pixels have four times the 
10 exposure time and this produces four times the amplitude. Effective 
sensitivity will therefore be increased fourfold and thus could allow for a 
reduction in filter size over known blue and red filters. 

So for this embodiment, the relative areas are four Y areas each at 1/8 of the 
15 overall size, one blue at 2/8 of the overall area and one red pixel again at 2/8 
of the overall area. YUV is then given by the following set of equations: 



Luminance for pixel 1 


Y1 


= Greyl 


-2R 


-3B 


20 Luminance for pixel 2 


Y2 


= Grey2 


-2R 


-3B 


Luminance for pixel 3 


Y3 


= Grey3 


-2R 


-3B 


Luminance for pixel 4 


Y4 


= Grey4 


-2R 


-3B 



Blue chrominance U = 3B - (greyl + grey2 + grey3 + grey4)/8 

Red chrominance V = 6R - 0.9 x (greyl + grey2 + grey3 + grey4)/4 

25 

The reduction in signal amplitude for red and blue pixels is not significant, with 
1/6 of full scale 12 bit (256), providing A/D values 0 to 42; this is particularly 
so given that image compression algorithms reduce resolution below this 
during quantisation. However, to correct for this there could be an increase in 
30 size of red and blue pixels, and for example doubling their filter areas would 
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increase A/D range to 84 and affect sensitivity by 16 times that of standard 
RGB filtered sensors. 

The above estimates are model values ignoring fill factor, which is concerned 
5 with the amount of associated silicon in the sensor chip dedicated to 
processing and which will effectively reduce as a result of this method. If grey 
pixels are same size as black/white, and fill factor is 25% for the latter, then 
the table below outlines the unit area for each sensor block. 

10 

black/white GreyRB 
Pixels 4 8 

Associated local circuitry 12 1 8 
Total area for GreyRB 16 26 
1 5 Absolute fill factor 25% 31 % 

Corrected fill factor* 1 5% 

* allowing for the fact that only four of the eight pixels are for luminance 

20 Therefore 25 fold increase in sensitivity is likely to be 15%/25% x 25 = 15:1 
increase in sensitivity. The likely increase in silicon could be 26/16 times the 
size, ie. 62% larger. However, the sensor array is only around 50% of the total 
silicon area, and therefore this equates to a 31% area increase. 

25 Figure 2 illustrates an alternative embodiment in which the red and blue filters 
of the block (20) are provided in strip like bands separating pairs of Y areas 
disposed above and below the bands. It is envisaged that the manufacture of 
such a filter structure would be much simplified because the ease of forming 
the red and blue bands would be increased because of less stringent 

30 alignment requirements of colour filters in relation to the pixel sensors. The 
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manner in which this embodiment works is similar to the embodiment of 
Figure 1, although because of the different filter structure the specific 
mathematics of the filter sizes and sensitivity calculations are different. Such 
calculations being readily derivable by those skilled in the art based on the 
5 equations provided above. 

A third embodiment (30) of the present invention is illustrated in Figure 3. In 
this embodiment green (34) and red (32) stripped filters are disposed 
lengthwise across the Y areas (and grey pixels) which are disposed in a 

10 generally square configuration as in the previously described embodiments. 
This embodiment is similar to that of either of Figures 1 or 2 in that for each 
block there are four grey sensors disposed in a square configuration around 
centrally disposed red and blue colour filters, the addition being that of the 
green and red stripped filters across the Y areas. Whilst the grey sensors in 

15 this embodiment still take a direct measure of luminance they are corrected 
for red and blue by the green and red strips and therefore provide a measure 
of luminance without the need for any calculation as required in embodiments 
for Figure 1 and 2. In other words, the green and blue strips correct for the 
influence of the central red and blue filters and accordingly yield direct Y 

20 measurements. 

The present invention may be embodied in other specific forms without 
departing from its essential attributes. For example, different specific 
configurations of the various colour/non-colour areas are possible giving 
25 analogous results. Accordingly reference should be made to the appended 
claims and other general statement's herein rather than to the foregoing 
specific description as indicating the scope of invention. 

Furthermore, each feature disclosed in this specification (which term includes 
30 the claims) and/or shown in the drawings may be incorporated in the invention 



independently of other disclosed and/or illustrated features. In this regard, the 
invention includes any novel features or combination of features disclosed 
herein either explicitly or any generalisation thereof irrespective of whether or 
not it relates to the claimed invention or mitigates any or all of the problems 
addressed. 

The appended abstract as filed herewith is included in the specification by 
reference. 
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Claims 

1 . An image sensing device having an array of image sensors and a filter 
structure disposed in relation to the image sensors, the filter structure 

5 comprising blocks of filter groups, each group having a combination of one or 
more areas adapted so as to allow light to pass therethrough onto 
corresponding image sensors with colour filtering, and one or more areas 
adapted so as to allow light to pass therethrough onto corresponding image 
sensors without colour filtering. 

10 

2. An imaging device according to claim 1, wherein said one or more 
areas adapted so as to allow light to pass therethrough onto corresponding 
image sensors with colour filtering comprise two colour filters. 

15 3. An imaging device according to claim 2, wherein the two colour filters 
are one each of red and blue filters. 

4. An imaging device according to any preceding claim, wherein said one 
or more areas adapted so as to allow light to pass therethrough onto 

20 corresponding image sensors without colour filtering comprise four filterless 
areas. 

5. An imaging device according to any preceding claim, wherein said one 
or more areas adapted so as to allow light to pass therethrough onto 

25 corresponding image sensors without colour filtering comprise four non-colour 
filters. 

6. An imaging device according to any preceding claim, wherein the 
arrangement of said areas in each block of filter groups in the filter structure 

30 comprises stripline formation of said respective areas. 
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7. An imaging device according to any preceding claim, wherein the 
image sensors are CMOS sensors. 

5 8. An imaging device according to any to any preceding claim, wherein 
said one or more areas adapted so as to allow light to pass therethrough onto 
corresponding image sensors without colour filtering include one or more 
colour filters disposed at least partially thereover. 

10 9. An imaging device according to claim 8, wherein said one or more 
colour filters disposed over the one or more areas adapted to allow light to 
pass therethrough without colour filtering comprise colour filter strips. 

10. An image sensing device according to any preceding claim, wherein 
15 said colour filtering areas are disposed centrally in the filter group and said 

areas without colour filtering are disposed in a generally square like 
configuration around the central colour filters. 

11. An imaging device having a filter structure substantially as herein 
20 before described with reference to and as shown in the accompanying 

drawings. 



12. A filter structure substantially as herein before described with reference 
to and as shown in the accompanying drawings. 



Abstract 



12 



This invention discloses a filter structure for a video image sensing device. 
The filter structure consists of a red colour filter and a blue colour filter and 
these are combined in a sensing block with non-colour or apertured windowed 
sensors i.e. grey sensors. 
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